feat: implement cronjob based sitemap automation#374
feat: implement cronjob based sitemap automation#374amaan-bhati wants to merge 12 commits intomainfrom
Conversation
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Implements cron-based sitemap generation and publication by moving sitemap creation out of the Next.js runtime and into scheduled GitHub Actions, with optional Google Search Console submission on updates.
Changes:
- Add
scripts/generate-sitemap.mjsto generatepublic/sitemap.xmlfrom WPGraphQL. - Replace SSR
/sitemap.xmlhandler by committing a staticpublic/sitemap.xml. - Add GitHub Actions workflows to (1) periodically sync
public/sitemap.xmlvia PR and (2) submit the sitemap to Google on main updates.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/generate-sitemap.mjs |
New Node script to fetch WP posts and build/write public/sitemap.xml. |
public/sitemap.xml |
New committed sitemap output (generated content). |
pages/sitemap.xml.tsx |
Removed SSR sitemap endpoint implementation. |
package.json |
Adds generate:sitemap script entry. |
.github/workflows/sync-sitemap.yml |
Scheduled job to regenerate sitemap and open/update a PR when it changes. |
.github/workflows/submit-google-sitemap.yml |
Submits sitemap to Google Search Console when public/sitemap.xml changes on main. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…view Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
nehagup
left a comment
There was a problem hiding this comment.
We don't need to create a new PR for sitemap changes. But we can trigger the workflow on request for sitemap edition on the same PR. So once it's merged it's part of the same PR.
|
Implementation wise, great direction.
If a single post has no modified field, the script assigns it the latest date of any post in the entire
|
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
Thanks @nehagup for the review and helping me out to optimise the solution, I have gone through all the points mentioned and addressed them: 1. Fallback lastmod - Removed the batch 2. Failure notifications - Acknowledged, skipping for now, will revisit when we have discussed the webhook set up and implementation. 3. Hand-rolled JWT - Replaced the 60-line bash/python/openssl block with the 4. next-sitemap - Confirmed it is not in the codebase; nothing to remove. 5. Priority — Updated to 6. Image sitemap - Implemented. Added curl -sS -X POST "https://wp.keploy.io/graphql" \
-H "Content-Type: application/json" \
-d '{"query":"{ posts(first: 50, where: { orderby: { field: MODIFIED, order: DESC } }) { edges { node { slug featuredImage { node { sourceUrl altText } } } } } }"}'50/50 posts returned absolute |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: amaan-bhati <amaanbhati49@gmail.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 7 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Replaced the runtime sitemap with a committed static sitemap and adds two workflows:
sync-sitemap.yml: regeneratespublic/sitemap.xmlfrom WordPress and opens/updates a PR if it changedsubmit-google-sitemap.yml: submitshttps://keploy.io/blog/sitemap.xmlto Google Search Console after sitemap changes land onmainWhat Changed
scripts/generate-sitemap.mjspublic/sitemap.xmlpublic/sitemap.xmlas the versioned sitemap source of truth.github/workflows/sync-sitemap.yml.github/workflows/submit-google-sitemap.ymlpages/sitemap.xml.tsxnext.config.js/sitemap.xml: 5 minutes/robots.txt: 24 hoursWhy
mainFlow
Sync sitemap
node scripts/generate-sitemap.mjsfetches WordPress posts and rewritespublic/sitemap.xml.git diffshows changes, the workflow creates or updates a PR onautomation/sync-sitemap.Submit to Google
main.submit-google-sitemap.ymlruns whenpublic/sitemap.xmlchanges.https://keploy.io/blog/sitemap.xmlto Search Console.Key Details
lastmodis normalized toYYYY-MM-DDmodifiedfall back to the latest valid postlastmodacross the sitemapRequired Config
Generator / sync workflow
WORDPRESS_API_URLhttps://wp.keploy.io/graphqlWORDPRESS_API_URLGoogle submission secrets
GOOGLE_CLIENT_IDGOOGLE_CLIENT_SECRETGOOGLE_REFRESH_TOKENGOOGLE_SEARCH_CONSOLE_SITE_URLExample:
Validation
xmllintnpm run lintnpm run buildSources
External: